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Claims 



What is claimed is: 



1 1 . A method for recognizing a state based on a current output, comprising: 

2 (a) retrieving current output from a remote application utilizing a network, the 

3 current output corresponding to a state of the application; 

4 (b) generating an output structure description of the state; 

5 (c) comparing the output structure definition to a pre-defined state definition of a 

6 recorded state; and 

7 (d) determining whether the state of the remote application corresponds to the 

8 recorded state based on the comparison. 

1 2. The method as recited in claim 1 ? wherein the output structure definition 

2 includes at least one of: a number of windows presented in the state, a number of 

3 frames in each window, a nesting structure of the frames, and a nesting structure 

4 of the windows. 

1 3. The method as recited in claim 1, wherein the recorded state is part of a pattern 

2 pre-recorded by a user. 

1 4. The method as recited in claim 1 , further comprising recognizing content of the 

2 remote output based on recorded content properties. 
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The method as recited in claim 4, wherein the recognized content is utilized for 
recognizing the state based at least in part on a weighted average of content 
recognition and content importance. 

The method as recited in claim 4, wherein the content is at least one document. 

The method as recited in claim 4, wherein the content is at least one content item 
of a document. 

The method as recited in claim 1, further comprising returning predefined 
information about the state if the state corresponds to the recorded state, the 
predefined information being for allowing extraction of data from the remote 
application. 

The method as recited in claim 1, wherein multiple state definitions are 
compared to the output structure, wherein each of the definitions has a 
weighting based on a probability that the state definition occurs when in another 
state, wherein the output structure is compared to the state definition having the 
highest probability of matching the state of the remote application. 

The method as recited in claim 9, wherein the weighting is based on a state 
resulting from execution of a prior action. 

A computer program product for recognizing a state based on a current output, 
comprising: 

computer code for retrieving current output from a remote application utilizing a 
network, the current output corresponding to a state of the application; 
computer code for generating an output structure description of the state; 
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computer code for comparing the output structure definition to a pre-defined 
state definition of a recorded state; and 

computer code for determining whether the state of the remote application 
corresponds to the recorded state based on the comparison. 

A system for recognizing a state based on a current output, comprising: 

logic for retrieving current output from a remote application utilizing a network, 

the current output corresponding to a state of the application; 

logic for generating an output structure description of the state; 

logic for comparing the output structure definition to a pre-defined state 

definition of a recorded state; and 

logic for determining whether the state of the remote application corresponds to 
the recorded state based on the comparison. 

A method for recognizing a state based on a current output, comprising: 
retrieving current output from a remote application utilizing a network, the 
current output corresponding to a state of the application; 
generating an output structure description of the state, the output structure 
description including a structure of the output; 

comparing the output structure definition to a pre-defined state definition of a 
recorded state, wherein the recorded state is part of a pattern pre-recorded by a 
user; 

recognizing content of the remote output based on recorded properties; and 
determining whether the state of the remote application corresponds to the 
recorded state based on the comparison and the recognized content. 

The method as recited in claim 13, wherein the output structure definition 
includes at least one of: a number of windows presented in the state, a number of 
frames in each window, and a nesting structure of the frames. 



-64- 



The method as recited in claim 13, further comprising returning predefined 
information about the state if the state corresponds to the recorded state, the 
predefined information being for allowing extraction of data from the remote 
application. 

The method as recited in claim 13, wherein the recognized content is utilized for 
recognizing the state based at least in part on a weighted average of content 
recognition and content importance. 

The method as recited in claim 13, wherein the content is at least one document. 

The method as recited in claim 13, wherein the content is at least one content 
item of a document. 

The method as recited in claim 13, wherein multiple state definitions are 
compared to the output structure, wherein each of the definitions has a 
weighting based on a probability that the state definition occurs when in another 
state, wherein the output structure is compared to the state definition having the 
highest probability of matching the state of the remote application. 

The method as recited in claim 19, wherein the weighting is based on a state 
resulting from execution of a prior action. 

A computer program product for recognizing a state based on a current output, 
comprising: 

computer code for retrieving current output from a remote application utilizing a 
network, the current output corresponding to a state of the application; 
computer code for generating an output structure description of the state, the 
output structure description including a structure of the output; 
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7 


(c) 


computer code for comparing the output structure definition to a pre-defined 


8 




state definition of a recorded state, wherein the recorded state is part of a pattern 


9 




pre-recorded by a user; 


10 


(d) 


computer code for recognizing content of the remote output based on recorded 


11 




properties; and 


12 


(e) 


computer code for determining whether the state of the remote application 


13 




corresponds to the recorded state based on the comparison and the recognized 


14 




content. 




22. 


A system for recognizing a state based on a current output, comprising: 


£ 2 


(a) 


logic for retrieving current output from a remote application utilizing a network, 


i u 

rfi 3 




the current output corresponding to a state of the application; 


F! 4 


(b) 


logic for generating an output structure description of the state, the output 


5 




structure description including a structure of the output; 


S |; 

m 6 


(c) 


logic for comparing the output structure definition to a pre-defined state 


ry y 




definition of a recorded state, wherein the recorded state is part of a pattern pre- 


5 8 




recorded by a user; 
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(d) 


logic for recognizing content of the remote output based on recorded properties; 


10 




and 


11 


(e) 


logic for determining whether the state of the remote application corresponds to 


12 




the recorded state based on the comparison and the recognized content. 


1 


23. 


A method for recognizing a state of a remote application utilizing individual 


2 




document recognition, comprising: 


3 


(a) 


receiving a plurality of documents; 


4 


(b) 


attempting to recognize each of the documents as being associated with a 


5 




predetermined state of a transaction pattern; and 


6 


(c) 


identifying the state of the remote application based on the recognition of the 


7 




documents. 
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A computer program product for recognizing a state of a remote application 

utilizing individual document recognition, comprising: 

computer code for receiving a plurality of documents; 

computer code for attempting to recognize each of the documents as being 

associated with a predetermined state of a transaction pattern; and 

computer code for identifying the state of the remote application based on the 

recognition of the documents. 

A system for recognizing a state of a remote application utilizing individual 

document recognition, comprising: 

logic for receiving a plurality of documents; 

logic for attempting to recognize each of the documents as being associated with 
a predetermined state of a transaction pattern; and 

logic for identifying the state of the remote application based on the recognition 
of the documents. 

A method for recognizing a state based on a current output, comprising: 
retrieving current output from a remote application utilizing a network, the 
current output corresponding to a state of the application; 
generating an output structure description of the state; 
wherein the output structure definition includes at least one of: a number of 
windows presented in the state, a number of frames in each window, a nesting 
structure of the frames, and a nesting structure of the windows; 
comparing the output structure definition to multiple pre-defined state 
definitions of a recorded state; 

wherein the recorded state is part of a pattern pre-recorded by a user; 
wherein each of the definitions has a weighting based on a probability that the 
state definition occurs when in another state; 
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(g) 


wherein the weighting is based on a state resulting from execution of a prior 


14 




action; 


15 


(h) 


wherein the output structure is compared to the state definition having the 


16 




highest probability of matching the state of the remote application; 


17 


(i) 


recognizing a document of the remote output based on recorded properties; 


18 


(j) 


determining whether the state of the remote application corresponds to the 


19 




recorded state based on the comparison and the document recognition; 


20 


(k) 


wherein the recognized content is utilized for recognizing the state of the remote 






application based at least in part on a weighted average of document recognition 


22 




accuracy and document importance; and 


23 


0) 


returning predefined information about the state of the remote application if the 


24 




state corresponds to the recorded state, the predefined information being for 


25 




allowing extraction of data from the remote application. 
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